Image Decoding Method

ABSTRACT

There is provided an image decoding method for decoding a picture to be decoded by referencing an appropriate picture even when random access reproduction is performed. The method includes: a first storage step (S 100 -S 104 ) for successively decoding respective pictures contained in a stream and storing the pictures used for reference in a picture memory ( 103 ), thereby performing management; a judgment step (S 106 ) for judging whether a random access reproduction has been specified; a state modification step (S 108 ) used when it is judged that the random access reproduction has been specified, for modifying the management state for managing the pictures in the picture memory ( 103 ); and a storage step (S 100 -S 104 ) for successively decoding the respective pictures from the random access reproduction destination assuming that the modified management state is the initial state and storing the pictures used for reference in the picture memory ( 103 ), thereby performing management.

TECHNICAL FIELD

The present invention relates to an image decoding method for decodingand reproducing coded pictures in a stream.

BACKGROUND ART

Recently, with the arrival of the age of multimedia in which audio,video and the like are integrally handled, existing information mediasuch as newspapers, journals, TVs, radios and telephones and other meansthrough which information is conveyed to people has come under the scopeof multimedia. Generally speaking, multimedia refers to something thatis represented by associating not only characters but also graphics,audio and especially images and the like together. However, in order toinclude the aforementioned existing information media in the scope ofmultimedia, it appears as a prerequisite to represent such informationin digital form.

However, when estimating the amount of information contained in each ofthe aforementioned information media as the amount of digitalinformation, the information amount per character requires 1 to 2 byteswhereas the audio requires more than 64 Kbits (telephone quality) persecond, and when it comes to a moving picture, it requires more than 100Mbits (present television reception quality) per second. Therefore, itis not realistic for the information media to handle such an enormousamount of information as it is in digital form. For example, althoughvideo phones are already in the actual use via Integrated ServicesDigital Network (ISDN) which offers a transmission speed of 64 Kbits/sto 1.5 Mbits/s, it is impossible to transmit images on televisions andimages taken by cameras directly through ISDN.

This therefore requires information compression techniques, and forinstance, in the case of the videophone, moving picture compressiontechniques compliant with H.261 and H.263 standards recommended byInternational Telecommunication Union-Telecommunication StandardizationSector (ITU-T) are employed. According to the information compressiontechniques compliant with the MPEG-1 standard, image information as wellas audio information can be stored in an ordinary music Compact Disc(CD).

Here, Moving Picture Experts Group (MPEG) is an international standardfor compression of moving picture signals standardized by InternationalStandards Organization/International Electrotechnical Commission(ISO/IEC), and MPEG-1 is a standard to compress moving picture signalsdown to 1.5 Mbps, that is, to compress information of TV signalsapproximately down to a hundredth. The transmission rate within thescope of the MPEG-1 standard is set to about 1.5 Mbps to achieve themiddle-quality picture, therefore, MPEG-2 which was standardized withthe view to meet the requirements of high-quality picture allows datatransmission of moving picture signals at a rate of 2 to 15 Mbps toachieve the quality of TV broadcasting. In the present circumstances, aworking group (ISO/IEC JTC1/SC29/WG11) in charge of the standardizationof the MPEG-1 and the MPEG-2 has achieved a compression rate which goesbeyond what the MPEG-1 and the MPEG-2 have achieved, further enabledencoding/decoding operations on a per-object basis and standardizedMPEG-4 in order to realize a new function required in the era ofmultimedia. In the process of the standardization of the MPEG-4, thestandardization of encoding method for a low bit rate was aimed;however, the aim is presently extended to a more versatile coding ofmoving pictures at a high bit rate including interlaced pictures.

Furthermore, MPEG-4 AVC and ITU H.264 have been standardized since 2003as a next-generation picture coding scheme with higher compression rate,which are jointly worked by the ISO/IEC and the ITU-T (for example,refer to Non-Patent Reference 1). Currently, regarding H.264 standard, adraft of its revised standard in compliance with a High Profile which issuited for High Definition (HD) pictures have been developed. Whereas aDVD player which reproduces a movie and the like are widely known as anapplication of using the MPEG-2 moving picture compression technology,the H.264 compression standard has been scheduled to be adapted to aplayer using a Blu-ray Disk ROM (BD-ROM). Currently, its format standardhas been developed.

In general, in coding of a moving picture, the amount of information iscompressed by reducing redundancy in temporal and spatial directions.Therefore, an inter-picture prediction coding, which aims at reducingthe temporal redundancy, estimates a motion and generates a predictivepicture on a block-by-block basis with reference to prior and subsequentpictures, and then codes a differential value between the obtainedpredictive picture and a current picture to be coded. Here, a “picture”is a term to represent a single screen and it represents a frame whenused for a progressive picture whereas it represents a frame or fieldswhen used for an interlaced picture. The interlaced picture here is apicture in which a single frame consists of two fields respectivelyhaving different time. For coding and decoding an interlaced picture,three ways are possible: processing a single frame either as a frame, astwo fields or as a frame/field structure depending on a block in theframe.

A picture to which an intra-picture prediction coding is performedwithout reference pictures is referred to as an “I-picture”. A pictureto which the inter-picture prediction coding is performed with referenceto only a single picture is referred to as a “P-picture”. A picture towhich the inter-picture prediction coding is performed by referringsimultaneously to two pictures is referred to as a “B-picture”. TheB-picture can refer to two pictures, arbitrarily selected from thepictures which are displayed either before or after a current picture tobe coded, as an arbitrary combination. Whereas the reference picturescan be specified for each block that is a fundamental unit of coding anddecoding, they are distinguished as a first reference picture and asecond reference picture. Here, the first reference picture is a firstreference picture to be described in a coded bit stream and the secondreference picture is a reference picture to be described after the firstreference picture in the coded bit stream. However, the referencepictures need to be already coded and decoded as a condition to code anddecode these I-picture, P-picture, and B-picture.

A motion compensation inter-picture prediction coding is used for codingthe P-picture or the B-picture. The motion compensation inter-pictureprediction coding is a coding method which adopts motion compensation toan inter-picture prediction coding. The motion compensation is a methodof reducing the amount of data while increasing prediction precision byestimating an amount of motion (this is referred to as a motion vector,hereinafter) of each part in a picture not by simply predicting apicture from a pixel value of a reference frame, and by performingprediction in consideration of the estimated amount of data. Forexample, the amount data is reduced by estimating a motion vector of acurrent picture to be coded and coding a predictive difference between apredicted value which is shifted as much as the estimated motion vectorand the current picture to be coded. Since this method requiresinformation about the motion vector at the time of decoding, the motionvector is also coded and recorded or transmitted.

The motion vector is estimated on a macroblock basis. Specifically, amotion vector is estimated by fixing a macroblock of a current pictureto be coded, moving a macroblock of the reference picture within asearching range, and finding a position of the reference block which isapproximate to the standard block.

FIG. 1A and FIG. 1B are diagrams, each of which showing a configurationof a conventional MPEG-2 stream.

As shown in FIG. 1A and FIG. 1B, the MPEG-2 stream has a followinghierarchical structure. A stream is made up of a plurality of Group ofPictures (GOP). A moving picture can be edited and randomly accessed bysetting each GOP as a basic unit for coding. Each GOP is made up ofplural pictures, each of which is one of an I picture, a P picture and aB picture. In addition, each stream, GOP and picture includes asynchronous signal (sync) indicating a boundary between respective unitsand a header that is data commonly included in respective units.

FIG. 2 is a diagram showing a configuration of another conventionalstream.

This stream corresponds to a ]VT (H.264/MPEG-4 AVC) which has beencurrently developed for standardization by a joint cooperation withITU-T and ISO/IEC. In JVT, there is no concept of a header and commondata is placed in a head of the stream with a name of a parameter setPS. Furthermore, while there is no concept corresponding to GOP, arandomly-accessible unit, which corresponds to GOP, can be structured bydividing data with a special picture unit which can be decoded withoutrelying on other pictures. This unit is called as a random access unitRAU. As the parameter set PS, there are a picture parameter set PPSwhich is data corresponding to a header of each picture and a sequenceparameter set SPS corresponding to a header of a unit that is equal toor greater than GOP in the MPEG-2. Each picture has an identifierattached which indicates one of the picture parameter set PPS and thesequence parameter set SPS to which the picture refers. Specifically,plural picture parameter sets PPS and sequence parameter sets SPS thatare different from each other are coded only once, so that excess codingcaused by coding the same parameter sets (headers) several times foreach picture is omitted and the compression rate is increased, byindicating, with an identifier, one of the parameter sets to which acurrent picture refers. A picture number PN is an identification numberfor identifying a picture. Here, the picture number PN is a numberindicating a display order of a picture, and is different fromPictureNumber showing a decoding order as disclosed in the Non-PatentReference 1. The sequence parameter set SPS includes the maximum numberof reference-available pictures, picture size and the like. The pictureparameter set PPS includes a type of variable length coding (switchingbetween Huffman coding and arithmetic coding), an initial value ofquantization step, the number of reference pictures, and the like.

FIG. 3A and FIG. 3B are diagrams for explaining a reference statebetween GOPs used for the conventional MPEG-2 and the like.

FIG. 3A shows a predictive configuration among pictures in a Closed GOP.In this diagram, the diagonally shaded pictures are pictures to bereferred to by other pictures. Furthermore, each picture is arranged indisplay order. In the Closed GOP configuration, B pictures (B6 and B7),which are displayed before the displayed time of an InstantaneousDecoding Refresh (IDR) picture, can be predictive-coded with referenceonly to the IDR picture and cannot refer pictures that belong to adifferent GOP. In addition, FIG. 3B shows a predictive configurationamong pictures in an Open GOP. In this diagram, the diagonally shadedpictures are pictures to be referred to by other pictures as in the caseof FIG. 3A. Furthermore, each picture is arranged in display order. Inthe Open GOP configuration, B pictures (B6 and B7), which are displayedtemporally before an I picture (I8), can be predictive-coded withreference to pictures in the same GOP3 and a P picture (P5) in a GOP4which is positioned immediately before the GOP3.

In the MPEG-2, P pictures (P2, P5, P11, and P14) can be predictive-codedwith reference to only an I picture or a P picture displayed temporallyimmediately before the current P picture. Furthermore, B pictures (B0,B1, B3, B4, B6, B7, B9, B10, B12 and B13) can be predictive-coded withreference to one of an I picture and a P picture displayed temporallyimmediately before the current B, and a one of an I picture and a Ppicture displayed temporally immediately after the current B picture,and their arrangement order in a stream has been determined.

On the other hand, the H.264 standard introduces a very flexiblepredictive structure among pictures in order to significantly increasecoding efficiency (a compression rate). Specifically, a P picture is notrestricted to refer to only one picture displayed immediately before thecurrent P picture. A different reference picture for the P picture canbe selected for each coded block from among I pictures, P pictures or Bpictures despite the display order of these pictures, if those pictureshave been decoded and managed in a buffer for reference pictures.Similarly, B picture is not restricted to refer to only one picture eachdisplayed immediately before and after the B picture. Here, a differentset of two pictures can be selected for each coded block from among Ipictures, P pictures or B pictures despite the display order of thosepictures.

In a BD-ROM format standard, pictures can be random access units RAUseven in the case where the pictures are arranged in a stream having theOpen GOP structure as similar to the case of the conventional structure.However, following restrictions are for example set forpredictive-coding among pictures under the H.264 standard.

(1) Since there is a possibility that B pictures (B6 and B7), which aredisplayed temporally immediately before an I picture designated as arandom access reproduction start point, refer to pictures displayedtemporally before and after the B pictures. Therefore, the B picturesare not displayed at the time of random access reproduction.

(2) A picture displayed temporally after the I picture designated as arandom access reproduction start point must not refer to a picturedisplayed temporally before said I picture.

FIG. 4 is a block diagram of an image decoding apparatus which realizesa conventional image decoding method.

The image decoding apparatus shown in FIG. 4 includes a variable lengthdecoding unit 901, a motion compensation unit 902, a picture memory 903,an adding unit 904 and a conversion unit 905.

The variable length decoding unit 901 decodes a stream Str, and outputsa quantized value Qco, a relative index Ind, a picture type Pty, and amotion vector MV. The quantized value Qco, the relative index Ind andthe motion vector MV are respectively inputted into and decoded by thepicture memory 903, the motion compensation unit 902, and the conversionunit 905.

The conversion unit 905 inverse quantizes the quantized value Qco so asto reconstruct a frequency coefficient, further inverse-frequencyconverts the frequency coefficient into a pixel differential value, andoutputs the resultant to the adding unit 904.

The adding unit 904 adds the pixel differential value and a predictiveimage outputted from the motion compensation unit 902, and generates adecoded picture Vout. The generated decoded picture Vout is stored intothe picture memory 903. Herein, as in the case where plural pictures canbe used as reference pictures, each block requires a reference number (arelative index Ind) for specifying an identification number of a pictureto be referred to. Accordingly, by obtaining a correspondence between arelative index Ind and a picture number of each picture stored in thepicture memory 903, a reference picture can be specified based on therelative index Ind.

The picture memory 903 has a reference picture list, and referencepictures that are decoded and used as references are stored in thereference picture list. Furthermore, the reference picture list has alist (STRL=Short Term Reference List) for performing First In First Out(FIFO) management of reference pictures, and a list (LTRL=Long TermReference List) for managing whether or not to store, into the list, ordelete the explicitly specified reference pictures.

The motion compensation unit 902 extracts an optimum image region for apredictive image from the decoded picture stored in the picture memory903, based on the motion vector and relative index Ind detected throughthe aforementioned processing. The motion compensation unit 902 thengenerates the predictive image and outputs it to the adding unit 904.

Next, it shall be explained about a method of generating a picturenumber PN which is to be attached to each picture for specifying apicture based on the relative index Ind. Here, summaries of two methodsout of three picture number PN generation methods defined by the H. 264are explained as examples.

PICTURE NUMBER GENERATION METHOD EXAMPLE 1

FIG. 5 is an explanatory diagram of a picture number generation methodexample 1.

Hereinafter, the picture number generation method example 1 shall beexplained with reference to FIG. 5. In the generation method example 1,a picture number PN is generated by adding an offset value Msb managedas a variable in decoding processing and a difference value (a countvalue Isb) from the offset value Msb attached to each picture.

In FIG. 5, each picture Pic is one of I, P, and B picture, and isarranged in display order on a screen. However, in decoding order, it isassumed that pictures are decoded in the following order of IDR, . . . ,P3, B1, B2, P6, B4, B5, I9, B7, B8, and . . . . Both Msb1 and Msb2indicate values of offset values Msb, where Msb1=96 and Msb2=112.

As shown in a head of the picture Pic arrangement, an IDR picture isplaced, as an I picture to be decoded at first in the access unit AU, ata start point of a stream or in a place where there is no referencerelationship between a current GOP to be decoded and a GOP which ispositioned immediately before the current GOP. First, when the IDRpicture is decoded, an offset value Msb is initialized to 0 and thecount value Isb is also 0. Therefore, a picture number PN of said IDRpicture becomes 0. For example, when the count value Isb is 3 for thenext P picture, Msb+Isb=0+3=3 is obtained. Therefore, a picture numberPN for said P picture is 3.

Furthermore, when the count value Isb is 1 for the subsequent B picture,0+1=1 is obtained so that a picture number PN for said B picture is 1.In addition, when the count value Isb is 2 for the next subsequent Bpicture, 0+2=2 is obtained so that a picture number PN for that Bpicture is 2.

Here, when the count value Isb reaches a predetermined value (a maximumcount value Lsb) while repeating the aforementioned operations, themaximum count value Lsb is added to the offset value Msb and the countvalue Isb is managed so as not to exceed the maximum count value Lsb, inorder to prevent an increase in the number of bits of the count valueIsb and a decrease in coding efficiency. For example, in the case wherethe maximum count value Lsb=16, if the count value Isb reaches themaximum count value Lsb for six times, it indicates that the offsetvalue Msb has been updated to the value of 0+16×6=96.

Furthermore, when the picture P3 is decoded, in the case where theoffset value Msb has been updated to 96 and the count value Isb equalsto 12, the picture number PN of the picture P3 is obtained as 96+12=108.Similarly, for B1, B2, P6, B4 and B5 pictures (in decoding order), whenthe respective count values Isb are 10, 11, 15, 13 and 14, the picturenumbers PN are respectively 107, 108, 111, 109 and 110.

Furthermore, for I9, B7, B8, B10 and subsequent pictures, the countvalue Isb exceeds the maximum count value Lsb of 16 if the count valueIsb is continued to be increased. Therefore, the count value Isb iscontrolled so as to be the value less than 16 by updating the offsetvalue Msb as obtained by the offset value of Msb=Msb1+maximum countvalue Lsb=96+16=112. Accordingly, for the pictures I9, B7, B8 and B10,the respective count values Isb are 2, 0, 1 and 3, and the respectivepicture numbers PN are 114, 112, 113 and 115.

As described in the above, a picture number PN is generated using thecount value Isb, which is attached to each picture, and the offset valueMsb, which is updated and managed in decoding processing, and an orderof displaying on a screen and reference pictures can be managed usingthe generated picture number PN.

PICTURE NUMBER GENERATION METHOD EXAMPLE 2

FIG. 6 is an explanatory diagram of the picture number generation methodexample 2.

In the picture number generation method example 2 shown in FIG. 6, theoffset value FNO managed as a variable in decoding processing is addedto the frame number (a count value) fn attached to each picture, andgenerates a picture number PN by doubling the sum. It should be notedthat, in the case of an unreferenced picture, a picture number PN isgenerated for that picture by subtracting 1 from the calculated sum.

FNO1 and FNO2 are values of offset values FNO. In the example shown inFIG. 6, FNO1 is 96 and FNO2 is 112.

First, when an IDR picture is decoded, an offset value FNO isinitialized to 0, and a frame number fn becomes 0 as well as the picturenumber PN for said IDR picture. When the frame number fn is 1 for thenext B picture, 2×(offset value FNO+frame number fn)=2×(0+1)=2 isobtained and the picture number PN for said B picture becomes 2.

Furthermore, when the frame number fn is 2 for the subsequent B picture,2×(0+2)=4 is obtained, and the picture number PN for that B picturebecomes 4. When the frame number fn is 3 for the next subsequent Ppicture, 2×(0+3)=6 is obtained and the picture number PN for said Ppicture becomes 6.

In the case where the frame number fn of a current picture is smallerthan the frame number of a picture which is decoded temporallyimmediately before the current picture, the offset value FNO is updatedby adding the maximum frame number MFN to the offset value FNO.

This is a mechanism for reducing the amount of coded bits by restrictingan available value range for the frame number to be equal to or smallerthan a predetermined value, and for also indicating a large picturenumber PN. In FIG. 6, for example, it is indicated that the offset valueFNO is FNO1 when the frame number fn of a current picture became smallerthan the frame number of a picture that is decoded temporallyimmediately before the current picture for six times, and the offsetvalue FNO1 is updated to the state of FNO2 when the above situationoccurred for the seventh time.

For example, when a picture B1 in the picture Pic is decoded, if theoffset value FNO is updated to 96 and the frame number fn is 10, thepicture number PN of the picture B1 is obtained by 2×(96+10)=212.Similarly, in the case where pictures B2, P3, B4, B5 and P6 (in decodingorder) have respective frame numbers fn of 11, 12, 13, 14 and 15, theirpicture numbers PN are respectively 214, 216, 218, 220 and 222.

Furthermore, when a picture B7 is decoded, if the frame number fn is 0which is the value smaller than 15 that is the frame number of a picturepositioned immediately before the picture B7, the offset value FNO isupdated to FNO=FNO1+maximum frame number MFN=96+16=112. Accordingly, thepicture number PN of the picture B7 becomes 2×(112+0)=224. Similarly,when the pictures B8, I9 and B10 have respective frame numbers fn of 1,2 and 3, respective picture numbers PN of 226, 228 and 230 are obtained.

Thus, as similar to the case of the picture number generation methodexample 1, the picture number PN can be generated using a value of aframe number fn attached to each picture and an offset value FNO whichis updated and managed in decoding processing. Therefore, the referencepictures can be managed using the generated picture number. It should benoted that, in the picture number generation method example 2, thedecoding order is same as the display order.

-   Non-Patent Reference 1: ISO/IEC 14496-10, International Standard:    “Information technology—Coding of audio-visual objects—Parts 10:    Advanced video coding” (Dec. 1, 2003), pp. 82-100.

DISCLOSURE OF INVENTION Problems that Invention is to Solve

However, in the aforementioned conventional image decoding method, thereis a problem of decoding a current picture to be decoded with referenceto inappropriate pictures when random access reproduction is performed.

For example, when random access reproduction is performed by a BD-ROM,if a position of a stream designated as a start point of the randomaccess reproduction has an Open GOP structure, normal decodingprocessing cannot be performed solely based on the aforementioned BD-ROMformat restriction.

Such problem is not caused in a picture reference relationship ininter-picture motion estimation prediction under the conventional MPEG-2and the like. In such case, a picture to be referred to by a B pictureis restricted to one of an I picture and a Picture positionedimmediately before said B picture or one of an I picture and a P picturepositioned immediately after said B picture. Therefore, even when randomaccess reproduction for the Open GOP is performed, normal decodingprocessing starting from an I picture can be performed simply by notdecoding and displaying only the B picture temporally positioned beforethe I picture which is positioned at the reproduction start point indisplaying order.

However, in the case of H.264 standard, plural reference pictures can beselected. Therefore, in the case where random access reproduction(trick-play) starting from a Random Access Point (RAP) having an OpenGOP structure is performed, picture numbers cannot be generated normallysince reproduction information of pictures, which should have beenstored in the picture memory 103 through original normal consecutivereproductions, positioned prior to the reproduction start position donot exist.

Specifically, when random access reproduction is performed, themanagement of pictures on the reference picture list based on thepicture numbers is disturbed causing, as a result, a problem that eachpicture included in a stream cannot be decoded normally with referenceto an appropriate picture.

Hereinafter, there shall be provided two examples relating to a state inwhich a picture number cannot be normally generated when random accessreproduction is performed so that based effect is given on themanagement of the reference picture list and the selection of referencepictures become abnormal.

EXAMPLE OF MALFUNCTION USING PICTURE NUMBER GENERATION METHOD EXAMPLE 1

FIG. 7 is a diagram of explaining a management method of an STRL usingthe picture number generation method example 1. It should be noted thatFIG. 7 shows a state of the STRL managed by FIFO when following the flowof decoding shown in FIG. 5. Here, in order to simplify the explanation,it is assumed that each STRL manages four reference pictures. As similarto the case of FIG. 5, PN indicates a picture number and Pic indicates apicture to be decoded. FIG. 7 differs from FIG. 5 in that pictures Picare aligned in decoding order. It is assumed that RAP indicates a randomaccess point and a stream has an Open GOP structure. Furthermore, STRL(a) to (f) respectively indicate a state of an STRL managed by FIFO, andshows a state transition of STRLs along with decoding of pictures.

When a random access unit RAU has a Closed GOP structure, an IDR pictureis firstly decoded. Herein, each region managed by the STRL becomes allempty (a state shown as STRL (a)), and the picture number PN isinitialized to 0. Next, after the IDR picture is decoded, said IDRpicture is inserted into the STRL (a state shown as STRL (b)).Hereinafter, after a picture Pic which is necessary to be managed as anew reference picture is decoded, cancellation and addition of referencepictures are performed in accordance with the FIFO management in eachSTRL.

For example, when the picture P3 having the picture number PN of 108 isdecoded as a reference picture, a picture having a picture number PNwhich is smaller than 99 from the STRL becomes an unreferenced picture,and said picture P3 having the picture number PN of 108 is newly addedto the STRL (a state shown as STRL (c)). Furthermore, when pictures B1and B2 having the picture numbers PN of 106 and 107 are decoded asunreferenced pictures, and the picture P6 having the picture number PNof 111 is decoded as a reference picture, a picture Px having a picturenumber PN of 99 from the STRL becomes an unreferenced picture, and saidpicture P6 having the picture number PN of 111 is newly added to theSTRL as a reference picture (a state shown as STRL (d)).

Furthermore, when pictures B4 and B5 having respective picture numbersPN of 109 and 110 are decoded as unreferenced pictures, and the pictureI9 having the picture number PN of 114 is decoded as a reference pictureover the random access point RAP, a picture Py having the picture numberPN of 102 becomes an unreferenced picture from STRL, and said picture I9having the picture number PN of 114 is newly added to the STRL as areference picture (state shown as STRL (e)).

Following that, when pictures B7 and B8 having respective picturenumbers PN of 112 and 113 are decoded as unreferenced pictures, and thepicture P12 having the picture number PN of 117 is decoded as areference picture, a picture Pz having the picture number PN of 105becomes an unreferenced picture from STRL, and said picture P12 havingthe picture number PN of 117 is newly added to the STRL as a referencepicture (state shown as STRL (f)). As similar to the aforementionedprocess, the pictures B10 and B11 having respective picture numbers PNof 115 and 116 are decoded as unreferenced pictures, and referencepicture management is continuously performed.

Each reference picture managed by the STRL is specified with therelative index Ind. Therefore, when a current picture to be decoded is aP picture, reference pictures are re-arranged in one list for a one-wayreference, and when the current picture is a B picture, they arere-arranged in two lists for two-way references. L0_for_B11 _(—) n (L0list) and L1_for_B11 _(—) n (L1 list) are lists to be generated when apicture B11 having a picture number PN of 116 is decoded. In the casewhere the current picture is a B picture, a reference picture having apicture number which is smaller than a picture number of the currentpicture is firstly arranged in order of a picture position closer to thecurrent picture from among the reference pictures on the STRL, and areference picture having a picture number which is greater than thepicture number of the current picture is then arranged in order of apicture position closer to the current picture. Consequently, theaforementioned L0 list is generated. Furthermore, a reference picturehaving a picture number which is greater than the picture number of thecurrent picture is firstly arranged in order of a picture positioncloser to the current picture from among the reference pictures on theSTRL, and a reference picture having a picture number which is smallerthan the picture number of the current picture is then arranged in orderof a picture position closer to the current picture. Consequently, theaforementioned L1 list is generated. Reference pictures are determinedby designating one picture each from the respective lists based on therelative indexes Ind.

Furthermore, Refldx corresponds to a list of relative indexes Ind andshows each relative index Ind so as to correspond to the arrangements ofreference pictures on the respective L0 list and L1 list. For example,in the case of L0_for_B11 _(—) n, the picture I9 having the picturenumber PN of 114 is selected based on the relative index (index0), thepicture P6 having the picture number PN of 111 is selected based on therelative index (index1), the picture P3 having the picture number PN of108 is selected based on the relative index (index2), and the pictureP12 having the picture number PN of 117 is similarly selected based onthe relative index (index3).

However, in such picture number PN generation method, there is a casewhere the L0 list and L1 list cannot be managed appropriately whenrandom access is performed.

FIG. 8 is a diagram of explaining an example of causing malfunction dueto the list managed in the picture number generation method example 1.The example shown in FIG. 8 shows an example of the case where skipreproduction (random access reproduction) is performed from the pictureI9 immediately after a B picture having a picture number of 101, that isthe picture positioned four pictures ahead of the picture P3 shown inFIG. 7, is decoded. In FIG. 8, same reference numbers as in FIG. 7 areattached to indicate the same elements and overlapping explanations areomitted.

STRL (g) in FIG. 8 shows a state of the STRL when a picture number PN is101. Furthermore, STRL (h) shows a state of the STRL after pictures areskipped from the B picture having the picture number PN of 101 to thepicture 19 and the picture I9 is decoded. Here, when skip operation isperformed, skipped pictures such as pictures P3, B1 and B2 (diagonallyshaded pictures Pic in FIG. 8) are not decoded. Therefore, the STRL isupdated from the state of STRL(g), and the offset value Msb is retainedat the previous offset value Msb (96). Accordingly, picture numbers ofpictures I9, B7 and B8 are generated respectively as Msb+Isb=96+2=98,Msb+Isb=96+0=96, and Msb+Isb=96+1=97. However, since it is not necessaryto decode B pictures displayed temporally immediately before an Ipicture at the time of random access, the pictures B7 and B8 indicatedwith striped lines in FIG. 8 are not decoded. Further, picture numbersof pictures P12, B10 and B11 are also generated respectively as 101, 99,and 100.

Furthermore, an L0 list and an L1 list are generated based on the valueof picture number PN attached to each picture and the picture number PNof a current picture to be decoded. Accordingly, in the case where thepicture B11 having a picture number PN of 100 is the current picture, L0list (L0_for B11 _(—) r 1) and L1 list (L1_for_B11 _(—) r 1) arerespectively “#99:Px, #98:I9, #101:P12, #102:Py” and “#101:P12, #102:Py,#99:Px, #98:I9”.

However, whereas the picture I9 can be referred to from the L0 listusing the relative index (index0) in the case of normal consecutivereproduction as shown in FIG. 7, the picture I9 cannot be referred toeven similarly using the relative index (index0) and the picture Px ismistakenly referred to in the case of random access reproduction asshown in FIG. 8. In other words, when random access reproduction isperformed, a current picture to be decoded is mistakenly decoded byreferring to an inappropriate picture.

EXAMPLE OF MALFUNCTION OF PICTURE NUMBER GENERATION METHOD EXAMPLE 2

FIG. 9 is a diagram of explaining a management method of an STRL by thepicture number generation method example 2.

In the case where the maximum value which can be managed as a picturenumber PN is a maximum picture number MPN, and a position of the pictureI0 is designated as a random access point RAP of an Open GOP, in thenormal consecutive reproduction, the offset value FNO at the time ofdecoding the picture I0 becomes a value obtained by the maximum picturenumber MPN−the maximum frame number MFN+1. Further, at the point whenpictures up to picture P8 are decoded, the picture number of the pictureP8 is the maximum picture number MPN so that the Open GOP is closed anda new GOP structure is started as a Closed GOP from the next IDRpicture. Furthermore, when pictures I0, B1, B2, . . . , B6, B7 and B8are respectively decoded, where respective frame numbers are 0, 1, 2, .. . , maximum frame number MFN−3, maximum frame number MFN−2, andmaximum frame number MFN−1, picture numbers PN of the respectivepictures are 2×(maximum picture number MPN−maximum frame number MFN+1),2×(maximum picture number MPN−maximum frame number MFN+2), 2×(maximumpicture number MPN−maximum frame number MFN+3), . . . , 2×(maximumpicture number MPN−2), 2×(maximum picture number MPN−1), and 2×(maximumpicture number MPN).

Here, it shall be explained about the case where a reproduction positionis jumped from the picture B4 toward back (Back Jump) to a random accesspoint RAP. In this case, the frame number fn of the picture B4 is themaximum frame number MFN−5, and the frame number fn of the picture I0 tobe decoded after the Back Jump is 0. Therefore, (frame number fn ofpicture B4)>(frame number fn of picture I0) is obtained. Accordingly,the frame numbers fn are decreased so that the offset value FNO isupdated from the maximum picture number MPN−maximum frame number MFN+1to a maximum picture number MPN+1 to which the maximum frame number MFNis added. Consequently, the picture number PN of the picture I0 becomes2(offset value FNO+frame number fn)=2(maximum picture numberMPN+1+0)=2(maximum picture number MPN+1).

Specifically, the picture number PN exceeds the maximum picture numberMPN which is the manageable maximum picture number so that there is apossibility that the picture number PN becomes a negative valuedepending on a bit precision of the picture number PN. If a negativevalue is given to the picture number PN, said picture number PN becomesthe smallest in the STRL. Therefore, as in the case of (malfunctionexample using picture number generation method example 1), L0 list andL1 list have states that are different from the state in the case ofnormal consecutive reproduction. As a result, a reference picture cannotbe specified normally using a relative index Ind so that malfunction inwhich subsequent decoding is not performed correctly, is caused.

Accordingly, regarding the aforementioned problem, an object of thepresent invention is to provide an image decoding method for decoding apicture to be decoded with reference to appropriate pictures even in thecase where random access reproduction is performed.

Means to Solve the Problems

In order to achieve the aforementioned object, an image decoding methodaccording to the present invention is an image decoding method fordecoding each of coded pictures in a stream with reference to a picturethat is previously decoded and stored in a memory, the method including:a first storage step of sequentially decoding each of the coded picturesin the stream, storing, into the memory, a picture to be used as areference from among the decoded pictures, and managing the picturestored in the memory; a determination step of determining whether or notrandom access reproduction has been designated; a state change step ofchanging a management state for managing the picture in the memory inthe case where it is determined that random access reproduction has beendesignated in the determination step; and a second storage step ofsequentially decoding each of the coded pictures from a random accessreproduction destination with the management state changed in the statechange step as an initial state, storing, into the memory, a picture tobe used as a reference from among the decoded pictures, and managing thepicture stored in the memory. For example, the coded pictures in thestream are divided into groups on an access unit basis, and in thedetermination step, it is determined whether or not random accessreproduction starting from an access unit which is allowed to refer to apicture of another access unit has been designated. Furthermore, in therespective first and second storage steps, the picture to be used as areference is managed by generating picture numbers for identifying eachof the decoded pictures and assigning a picture number to each of thedecoded pictures, and in the state change step, an offset value forgenerating the picture number is updated as the management state.

With the conventional technology, in the case where random accessreproduction is designated, the offset value is not sometimes updated oris sometimes updated to an unnecessarily greater value so that aninappropriate picture is generated. Therefore, a picture to be storedinto a memory cannot be appropriately managed using a correspondingpicture number. Accordingly, in the image decoding method according tothe present invention, in the case where random access reproduction isdesignated, a management state for managing a picture such as a previousoffset value is changed and random access reproduction is performed withthe changed management state as an initial state. Therefore, anappropriate picture number is generated and a picture to be stored intoa memory can be correctly managed. As a result, each picture to bedecoded from random access reproduction destinations can be decoded withreference to an appropriate picture. Accordingly, in the BD-ROM formatstandard using H.264 standard, both of normal reproduction and randomaccess reproduction can be correctly performed.

Furthermore, in the first storage step, the offset value may be updatedso that the offset value is increased every time when all the picturesincluded in a current access unit to be decoded are decoded, and apicture number for each of the decoded pictures may be generated usingthe offset value and a count value assigned to each of the decodedpictures in the stream on an access unit basis; in the state changestep, the offset value may be updated so that the offset value isincreased; and in the second storage step, with the offset value updatedin the state change step as an initial state, the offset value may beupdated so that the offset value is increased every time when all thepictures in a current access unit to be decoded are decoded, and apicture number for each of the decoded pictures from the random accessreproduction destination may be generated using the count value and theoffset value.

Accordingly, in the case where random access reproduction is designated,the offset value is updated to a significantly greater value as theoffset value is increased every time when all pictures included in anaccess unit are decoded. Therefore, it can be prevented that a picturenumber of a current picture as a random access reproduction destinationbecomes smaller than the picture numbers of the pictures prior to thecurrent picture, so that pictures stored in the memory can be correctlymanaged.

For example, in the respective first and second storage steps and statechange step, the offset value is updated so that the offset value isincreased at least by the maximum value of a count value assigned on anaccess unit basis.

Accordingly, it can be reliably prevented that the picture number of thecurrent picture as a random access reproduction destination becomessmaller than the picture numbers of the pictures prior to the currentpicture.

Furthermore, in the respective first and second storage steps, thepicture to be used as a reference may be managed by generating picturenumbers for identifying each of the decoded pictures and assigning apicture number to each of the decoded pictures, and in the state changestep, the management state may be updated by changing all the picturesstored in the memory to unreferenced pictures.

Accordingly, in the case where random access reproduction is designated,all pictures which are previously stored in the memory are changed tounreferenced pictures. Therefore, it can be prevented to causeinstability in management based on picture numbers between a picturestored in the memory before the random access reproduction is performedand a picture to be stored into the memory after the random reproductionaccess reproduction is performed. As a result, each picture to bedecoded from the random access reproduction destinations can be decodedwith reference to an appropriate picture.

Furthermore, in the respective first and second storage steps, thepicture to be used as a reference may be managed by generating picturenumbers for identifying each of the decoded pictures and assigning apicture number to each of the decoded pictures; in the first storagestep, an offset value for generating a picture number may be updated sothat the offset value is increased every time when a count value of acurrent picture to be decoded is smaller than a count value of a picturedecoded immediately before the current picture, and a picture number foreach of the decoded pictures may be generated using the offset value andthe count value assigned to each of the decoded pictures in the streamon an access unit basis; in the state change step, the offset valueexisting as the management state may be updated so that the offset valueis decreased; and in the second storage step, with the offset valueupdated in the state change step as an initial state, the offset valuemay be updated so that the offset value is increased, every time when acount value of a current picture to be decoded is smaller than a countvalue of a picture decoded immediately before the current picture, and apicture number for each of the decoded pictures from the random accessreproduction destination may be generated using the count value and theoffset value. For example, in the state change step, the offset value isupdated to 0.

With the conventional technology, in the case where random accessreproduction is designated, an inappropriate picture number is generatedby updating the offset value to an insignificantly greater value so thata picture to be stored into a memory cannot be appropriately managedusing a corresponding picture number. Accordingly, in the image decodingmethod according to the present invention, in the case where randomaccess reproduction is designated, previous offset values are changed tothe smaller values, pictures in the memory are changed to unreferencedpictures, and random access reproduction is performed with the changedmanagement state as an initial state. Therefore, an appropriate picturenumber is generated and the picture to be stored into the memory can becorrectly managed. As a result, each picture to be decoded from therandom access reproduction destinations can be decoded with reference tothe appropriate picture. In other words, when random access reproductionis designated, the offset value is updated to a smaller value so that ageneration of instability in management based on the picture number dueto an abnormal increase in the picture number can be prevented.

Furthermore, further in the second storage step, in the case where thereis an unreferenced picture in the memory, the unreferenced picture maybe replaced with a reference permissible picture which is stored mostrecently.

Accordingly, further appropriate picture can be referred to.

Furthermore, further in the second storage step, a picture to be storedinto the memory may be managed using start information based on a startof random access reproduction. For example, in the second storage step,a random access number which is to be updated every time when randomaccess reproduction is started may be used as the start information.

With the conventional technology, the picture of the oldest display timeis specified and deleted from the memory so that there is a problem thatpictures are deleted even they are not displayed when many of the randomaccess reproductions are designated. Accordingly, in the image decodingmethod according to the present invention, pictures that have not beendisplayed can be kept in the memory by managing the pictures usingrandom access numbers.

Furthermore, the memory includes a first storage region and a secondstorage region, each having a different picture management mode, andfurther in the state change step, a size of the first storage region anda size of the second storage region may be changed complementarilyaccording to a random access destination. For example, in the firststorage region, an oldest picture which has been stored for a longesttime is deleted when a new picture is stored, and in the second storageregion, a designated picture is deleted when a new picture is stored.

Accordingly, it can be prevented that the size of the first storageregion and the size of the second storage region differ for normalconsecutive reproduction and random access reproduction, so thatpictures to be stored into the memory can be correctly managed.

Furthermore, in the respective first and second storage steps, thepicture to be used as a reference may be managed by generating picturenumbers for identifying each of the decoded pictures and assigning apicture number to each of the decoded pictures, and in the state changestep, a picture number of a picture stored in the memory may be changedto a picture number which is smaller than a picture number of a picturefrom the random access reproduction destination.

Accordingly, as described in the above, there is a state as if a picturethat has not been decoded by random access reproduction is normallydecoded and stored in the memory. As a result, each picture to bedecoded from the random access reproduction destination can be decodedwith reference to appropriate pictures.

Furthermore, in the state change step, the picture stored in the memorymay be replaced with a dummy picture having a picture number which issmaller than the picture number of the picture from the random accessreproduction destination. For example, the dummy picture is a picturewhich is not included in the stream or a picture decoded in the firststorage step.

Accordingly, as described in the above, there is a state as if a picturethat has not been decoded by random access reproduction is normallydecoded and stored in the memory. As a result, each picture to bedecoded from the random access reproduction destination can be decodedwith reference to appropriate pictures.

It should be noted that the present invention can be realized as notonly such image decoding method, but also as an image decoding apparatusand an integration circuit which operate using such method, a programfor causing a computer to execute operations based on the method, and asa recording medium for storing such program.

EFFECTS OF THE INVENTION

The image decoding method of the present invention has an effect that apicture to be coded can be decoded with reference to an appropriatepicture even in the case of random access reproduction.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A is a diagram showing a configuration of a conventional MPEG-2stream.

FIG. 1B is a diagram showing another configuration of the conventionalMPEG-2 stream.

FIG. 2 is a diagram showing a configuration of other conventionalstream.

FIG. 3A is a diagram for explaining a reference state between GOPs usedin the conventional MPEG-2 and the like.

FIG. 3B is a diagram for explaining another reference state between GOPsused in the conventional MPEG-2 and the like.

FIG. 4 is a block diagram of an image decoding apparatus which realizesa conventional image decoding method.

FIG. 5 is a diagram for explaining a picture number generation methodexample 1.

FIG. 6 is a diagram for explaining a picture number generation methodexample 2.

FIG. 7 is a diagram for explaining an STRL management method using theimage number generation method example 1.

FIG. 8 is a diagram for explaining an example of causing malfunctionusing a list managed by the image number generation method example 1.

FIG. 9 is a diagram for explaining an STRL management method using theimage number generation method example 2.

FIG. 10 is a block diagram of an image decoding apparatus which realizesan image decoding method according to the first embodiment of thepresent invention.

FIG. 11 is a flowchart showing an example of operations by the imagedecoding apparatus according to the first embodiment of the presentinvention.

FIG. 12 is a flowchart showing operations by an image number generationunit according to the first embodiment of the present invention.

FIG. 13 is a diagram for explaining a state of a picture memoryaccording to the first embodiment of the present invention.

FIG. 14 is a diagram for explaining a method of generating an L0 listaccording to a first variation of the first embodiment of the presentinvention.

FIG. 15 is a diagram for explaining a method of generating an L0 listand an L1 list according to a second variation of the first embodimentof the present invention.

FIG. 16 is a block diagram of an image decoding apparatus which realizesan image decoding method according to a second embodiment of the presentinvention.

FIG. 17 is a flowchart showing an example of operations by the imagedecoding apparatus according to the second embodiment of the presentinvention.

FIG. 18 is a flowchart showing operations of a picture number generationunit according to the second embodiment of the present invention.

FIG. 19 is a diagram for explaining a method of managing a referencepicture list (STRL) according to the first variation of the secondembodiment of the present invention.

FIG. 20A is a flowchart showing an operation that the memory managementunit assigns a RAID number to a picture to which the RAID number is tobe assigned, according to the first variation of the second embodimentof the present invention.

FIG. 20B is a flowchart showing an operation that the memory managementunit selects a picture to be deleted from among pictures stored in thereference picture list, according to the first variation of the secondembodiment of the present invention.

FIG. 21A is a diagram showing a state of a reference picture listaccording to a second variation of the second embodiment of the presentinvention.

FIG. 21B is a diagram showing other state of a reference picture listaccording to the second variation of the second embodiment of thepresent invention.

FIG. 21C is a diagram showing other state of the reference picture listaccording to the second variation of the second embodiment of thepresent invention.

FIG. 21D is a diagram showing other state of the reference picture listaccording to the second variation of the second embodiment of thepresent invention.

FIG. 21E is a diagram showing other state of the reference picture listaccording to the second variation of the second embodiment of thepresent invention.

FIG. 21F is a diagram showing other state of the reference picture listaccording to the second variation of the second embodiment of thepresent invention.

FIG. 22A is a diagram for explaining the case of realizing the imagedecoding method according to the first and second embodiments in acomputer system using a program recorded on a recording medium such as aflexible disc.

FIG. 22B is other diagram for explaining the case of realizing the imagedecoding method according to the first and second embodiments in acomputer system using a program recorded on a recording medium such as aflexible disc.

FIG. 22C is other diagram for explaining the case of realizing the imagedecoding method according to the first and second embodiments in acomputer system using a program recorded on a recording medium such as aflexible disc.

FIG. 23 is a diagram showing a structure of an AV processing apparatusaccording to a fourth embodiment.

NUMERICAL REFERENCES

100 image decoding apparatus

101 variable length decoding unit

102 motion compensation unit

103 picture memory

104 adding unit

105 conversion unit

106 memory management unit

107 picture number generation unit

108 reproduction determination unit

Pic Picture

RAP Random Access Point

PN Picture Number

STRL Short Term Reference List

LTRL Long Term Reference List

RPL Reference Picture List

Str Stream

Qco Quantized Value

Ind Relative Index

Pty Picture Type

MV Motion Vector

Vout Reconstructed Picture

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, it shall be explained about an image decoding method of thepresent invention with reference to drawings.

First Embodiment

In an image decoding method according to the first embodiment of thepresent invention, a picture number is generated based on a picturenumber generation method example 1 shown in FIG. 5, and decodes eachpicture in a stream.

FIG. 10 is a block diagram of an image decoding apparatus which realizesthe image decoding method according to the present embodiment.

An image decoding apparatus 100 according to the present embodimentincludes a variable length decoding unit 101, a motion compensation unit102, a picture memory 103, an adding unit 104, a conversion unit 105, amemory management unit 106, a picture number generation unit 107, and areproduction determination unit 108.

The variable length decoding unit 101 decodes a stream Str, and outputsa quantized value Qco, a relative index Ind, a picture type Pty, and amotion vector MV. The quantized value Qco, the relative index Ind andthe motion vector MV are respectively inputted to the picture memory103, the motion compensation unit 102 and the conversion unit 105, anddecoded thereby.

The conversion unit 105 inverse quantizes the quantized value Qco,reconstructs it to a frequency coefficient, further performs inversefrequency conversion on the frequency coefficient to obtain a pixeldifferential value, and outputs the pixel differential value to theadding unit 104.

The adding unit 104 adds the pixel differential value and the predictivepicture outputted from the motion compensation unit 102, and generates adecoded picture Vout. The generated decoded picture Vout is stored intothe picture memory 103. Herein, in order to use plural pictures asreference pictures, each block requires a reference number (a relativeindex Ind) for specifying an identification number of a picture to bereferred to. Accordingly, reference pictures can be specified based onrelative indexes Ind by taking a correspondence between the relativeindex Ind and the picture number of each picture stored in the picturememory 103.

The motion compensation unit 102 extracts an optimum image region forthe predictive image from the decoded picture stored in the picturememory 103 using the motion vector and the relative index Ind detectedas the result of the aforementioned process. The motion compensationunit 102 then generates the aforementioned predictive image and outputsit to the adding unit 104.

The reproduction determination unit 108 determines a reproductionformat. Specifically, when a sequence is reproduced, the reproductiondetermination unit 108 determines whether or not to perform normalconsecutive reproduction or random access reproduction from a randomaccess point RAP having an Open GOP structure. For example, thereproduction determination unit 108 determines a reproduction format inaccordance with an instruction from a system control positioned at anupper hierarchy of the image decoding apparatus. The reproductiondetermination unit 108 also determines, in the case where a GOP of therandom access point RAP includes an IDR picture, that said GOP has aClosed GOP structure, and determines, in the case where the GOP does notinclude an IDR picture, that said GOP has an Open GOP structure. Itshould be noted that it is possible to estimate a reproduction formatfrom a result of tracing the decoding of frame numbers fn assigned torespective pictures. Furthermore, the normal consecutive reproductionincludes random access reproduction from the Closed GOP.

The reproduction determination unit 108 outputs a reproductiondetermination signal indicating such determination result.

The picture number generation unit 107 obtains a reproductiondetermination signal from the reproduction determination unit 108,generates a picture number based on the reproduction format (normalconsecutive reproduction or random access reproduction) indicated by theobtained reproduction determination signal, and outputs the picturenumber.

The memory management unit 106 manages a picture to be stored into thepicture memory 103 using the reproduction determination signal outputtedfrom the reproduction determination unit 108 and the picture numberoutputted from the picture number generation unit 107.

FIG. 11 is a flowchart which shows an example of operations by the imagedecoding apparatus 100.

First, when reproduction of a stream Str is started, the image decodingapparatus 100 decodes a current picture to be decoded which ispositioned at a start position (Step S100), and generates a picturenumber for the current picture based on the generation method example 1(Step S102). The image decoding apparatus 100 then assigns the generatedpicture number to the current picture, and stores the current pictureinto the picture memory 103 (Step S104). Next, the image decodingapparatus 100 determines whether or not random access reproduction isdesignated (Step S106), and when determining that random accessreproduction is designated (Y in Step S106), changes a management stateof the picture (Step S108). The image decoding apparatus 100 thenrepeatedly executes the processes from Step S100 to the pictures as therandom access reproduction destinations. Specifically, the imagedecoding apparatus 100 sequentially decodes each picture from the randomaccess reproduction destination when the changed management state as aninitial state, stores pictures to be used as references into the picturememory 103, and manages them.

On the other hand, when determining that random access reproduction isnot specified in Step S106 (N in Step S106), the image decodingapparatus 100 determines whether or not there is a not-yet-decodedpicture along with the sequence reproduction (Step S110). Whendetermining that there is a not-yet-decoded picture (Y in Step S110),the image decoding apparatus 100 repeatedly executes processes from StepS100 to the yet-to-be decoded picture. On the other hand, whendetermining that there is no not-yet-decoded picture (N in Step S110),reproduction processing of the stream Str is terminated.

FIG. 12 is a flowchart showing operations by the picture numbergeneration unit 107.

As shown in FIG. 12, the picture number generation unit 107 obtains areproduction determination signal from the reproduction determinationunit 108 (Step S120), and determines, based on the obtained reproductiondetermination signal, whether or not a reproduction format to bedesignated is normal consecutive reproduction or random accessreproduction from a random access point RAP of the Open GOP structure(Step S122).

Here, when determining that normal consecutive reproduction isdesignated in Step S122, the picture number generation unit 107generates a picture number using the generation method example 1.

On the other hand, when determining that random access reproduction isdesignated in Step S122, the picture number generation unit 107 adds amaximum count value Lsb to the offset value Msb, and updates the offsetvalue Msb (Step S126). The picture number generation unit 107 thengenerates, for a picture of the random access point RAP, a picturenumber PN obtained by adding the updated offset value Msb and the countvalue Isb of the picture, based on the generation method example 1 (StepS128).

Accordingly, in the present embodiment, the management state of apicture is changed when random access reproduction is performed.Specifically, the picture number PN at the time of the start of randomaccess reproduction is set to be a value greater than any picture numberPN of pictures Pic that are already included in the STRL, by adding themaximum count value Lsb to the offset value Msb so as to update theoffset value Msb. Consequently, occurrence of malfunction can beprevented.

Note that whereas, in the present embodiment, an offset value Msb isupdated by adding the maximum count value Lsb to the offset value Msb,the picture number PN at the time of the start of random accessreproduction may be set to a value which is greater than any picturenumber PN of the pictures Pic that are already included in the STRL.

FIG. 13 is a diagram for explaining a state of the picture memory 103managed according to the present embodiment.

First, when a picture I9 is decoded due to occurrence of skipping(random access reproduction) after a picture B having a picture numberof 101 is decoded, the offset value Msb is updated to 96+16=112. Next,Msb+Isb=112+2=114 is generated as a picture number PN for the pictureI9. As a result, the reference picture list is updated to a state ofSTRL (h′), and consecutively updated to a state of STRL (i′) after apicture P12 is decoded. Furthermore, a picture B10 and a picture B11 areconsecutively decoded.

Herein, the picture number PN of the picture P12 is the largest in theSTRL (a reference picture list), and the picture number PN of thepicture IP is the second largest. Therefore, an L0 list (L0_for_B11 _(—)r 2) and an L1 list (L1_for_B11 _(—) r 2) shown in FIG. 13 aregenerated. As a result, the picture I9 and the picture P12, whichconventionally could not specified with the same relative index Ind usedfor normal consecutive reproduction, can be specified.

As described in the above, in the present embodiment, when random accessreproduction is performed, the offset value Msb is updated at first sothat the offset value Msb is forcedly increased, and the managementstate of the picture is changed. Accordingly, it can be prevented thatthe picture number of a picture after the random access reproduction isperformed as shown in FIG. 8 becomes smaller than the picture number ofthe picture before the random access reproduction is performed.Consequently, occurrence of management instability based on the picturenumbers PN of pictures listed on the reference picture list isprevented, and as similar to the case of normal consecutivereproduction, a picture to be decoded can be decoded with reference toappropriate pictures using relative indexes Ind.

(First Variation)

Here, it shall be explained about a variation of the picture managementstate change according to the present embodiment.

In the present embodiment, when random access reproduction isdesignated, the picture management state is changed by updating theoffset value Msb. On the other hand, in the present variation, allpictures stored in the picture memory 103 as references are changed tounreferenced pictures.

Accordingly, only the pictures, which have been decoded since the timeof start of random access reproduction, are stored into the picturememory 103. Therefore, occurrence of confusion between pictures storedbefore the start of random access reproduction and pictures stored afterthe start of random access reproduction can be prevented. Furthermore,the picture number PN is not forcibly increased at the time of randomaccess reproduction so that it does not exceed the manageable maximumvalue (Maximum Picture Number MPN) of the picture number PN. Thus,occurrence of malfunction due to an increase of the picture number PNcan be prevented.

Here, special process may be further performed in the generation of anL0 list.

FIG. 14 is a diagram for explaining a method of generating an L0 list.

First, when skipping (random access reproduction) is occurred and apicture I9 is decoded after the picture B having the picture of number101 is decoded, all pictures on the reference picture list (STRL) areturned into unreferenced pictures. Specifically, pictures stored in thereference picture list are deleted as pictures for references, and theregions in which the deleted pictures had been stored become empty.Next, in accordance with the generation method example 1,Msb+Isb=96+2=98 is generated as a picture number PN for the picture I9.As a result, the reference picture list is updated to a state of STRL(h′) and then updated to a state of STRL (i′) after the picture P12 isdecoded. Furthermore, the picture B10 and the picture B11 areconsecutively decoded.

From the reference picture list in the state of such STRL (i′), an L0list (L0_for_B11 _(—) r 3) and an L1 list (L1_for_B11 _(—) r 3) aregenerated using a method determined by the aforementioned standard.

Furthermore, the L0 list (L0_for_B11 _(—) r 3) is updated to an L0 list(L0_for_B11 _(—) r 4) by consecutively inserting, into the emptyregions, pictures that are the same as the picture having the picturenumber PN which is lastly added.

Accordingly, a desired picture can be appropriately selected as areference picture from the L0 list even in the case of random accessreproduction, by inserting a picture into the empty region.

For example, assuming the case where the picture B11 refers to apreceding picture I9 and a subsequent picture P12. In this case, inorder to refer to these pictures I9 and P12, a 0th relative index(Index0) may be specified for the L0 list (L0_for_B11 _(—) r 4) and a0th relative index (Index0) may be specified for the L1 list (L1_for_B11_(—) r 3), as similar to the case of normal consecutive reproductionshown in FIG. 7.

Furthermore, for example, it is assumed the case where the picture B11refers to two pictures which are subsequent to the picture P12. In thiscase, in the normal consecutive reproduction shown in FIG. 7, it isnecessary to specify the third relative index (Index3) for the L0 list(L0_for_B11 _(—) n) in order to refer to the picture P12.

Here, if the empty region is kept empty without updating the L0 list(L0_for_B11 _(—) r 3), the picture P12 cannot be referred to even whenthe third relative index (Index3) is specified for the L0 list(L0_for_B11 _(—) r 3). Therefore, the L0 list (L0_for_B11 _(—) r 3) isupdated to the L0 list (L0_for_B11 _(—) r 4). Specifically, the pictureP12, which is stored immediately before the picture B11, is insertedinto the empty region on the L0 list (L0_for_B11 _(—) r 3). Accordingly,the picture P12 can be referred to by specifying the third relativeindex (Index3) as is the same in the case of normal consecutivereproduction shown in FIG. 7. Furthermore, by thus filling the emptyregion on the L0 list, the picture P12 can be normally referred to evenin the case where any one of first to third relative indexes Ind isspecified.

(Second Variation)

Here, it shall be explained about other variation regarding themanagement state change of a picture according to the presentembodiment.

In the present embodiment, when random access reproduction isdesignated, the picture management state is changed by updating theoffset value Msb. On the other hand, in the present variation, thepicture management state is changed by inserting, into the STRL, dummypictures having picture numbers which are smaller than a picture numberof a picture from the random access reproduction destination as many asthe number of reference-permissible pictures.

Accordingly, it looks as if pictures which were not decoded because ofrandom access are normally decoded and stored into the picture memory103. As a result, each picture to be decoded from the random accessreproduction destination can be decoded with reference to appropriatepictures.

FIG. 15 is a diagram for explaining a method of generating an L0 listand an L1 list according to the present variation.

First, after decoding the picture B having the picture number 101, whenskipping (random access reproduction) is occurred, a picture number PNto be assigned to the picture I9 is calculated before decoding thepicture I9. For example, in accordance with the generation methodexample 1, Msb+Isb=96+2=98 is calculated as the picture number PN forthe picture I9. Then, dummy pictures having picture numbers PN that aresmaller than the picture number PN of the picture I9 are inserted intothe reference picture list (STRL) as many as the number ofreference-permissible pictures. As a result, the reference picture listbecomes a state of STRL (j). In other words, a picture Py having thepicture number of 102 is replaced with a dummy picture having thepicture number of 97, and a picture Px having the picture number of 99is replaced with a dummy picture having the picture number of 96.Furthermore, a picture Pw having the picture number of 96 is replacedwith a dummy picture having the picture number of 95, and a picture Pvhaving the picture number of 93 is replaced with a dummy picture havingthe picture number of 94.

Here, the aforementioned dummy pictures are not actually used asreference pictures in the subsequent decoding process so that they maybe pictures, which are not included in the stream Str, to which picturenumbers which are smaller than the picture number of a picture as therandom access reproduction destination are assigned. For example, suchdummy pictures are pictures having pixel values, all of which arepredetermined values (for example, 0), or pictures which do not have anypixel value. Furthermore, the dummy pictures may be pictures that havebeen already decoded from among the pictures in the stream Str. Forexample, the dummy picture is a picture that has been decodedimmediately before a current picture to be decoded. Note that, insteadof replacing a picture stored in the reference picture list, only thepicture number of said picture may be changed to a picture number thatis smaller than the picture number of a picture as the random accessreproduction destination.

Next, when the picture I9 is decoded, the reference picture list isupdated to a state of STRL (h′), and then the reference picture list isupdated to a state of STRL (i′) continuously after the decoding of thepicture P12. Furthermore, the picture B10 and the picture B11 areconsecutively decoded.

From such reference picture list in the state of STRL (i′), an L0 list(L0_for_B11 _(—) r 5) and an L1 list (L1_for_B11 _(—) r 5) are generatedusing the method defined by the aforementioned standard.

Accordingly, by inserting dummy pictures, desired pictures can be moreappropriately selected as reference pictures from the L0 list even inthe case where random access reproduction is performed.

For example, it is assumed a case where the picture B11 refers to apreceding picture I9 and a subsequent picture P12. In this case, inorder to refer to these pictures I9 and P12, the 0th relative index(Index0) may be specified for the L0 list (L0_for_B11 _(—) r 5) and the0-th relative index (Index0) may be specified for the L1 list(L1_for_B11 _(—) r 5), as in the case of normal consecutive reproductionshown in FIG. 7.

Furthermore, for example, it is assumed the case where the picture B11refers to two pictures subsequent to the picture P12. In this case, inthe normal consecutive reproduction shown in FIG. 7, in order to referto the picture P12, the third relative index (Index3) is required to bespecified for the L0 list (L0_for_B11 _(—) n). As totally similar tothat case, the picture P12 can be referred to by specifying the thirdrelative index (Index3) for the L0 list (L0_for_B11 _(—) r 5).

Second Embodiment

In the image decoding method according to a second embodiment of thepresent invention, each picture included in a stream is decoded bygenerating a picture number based on the picture number generationmethod example 2 shown in FIG. 6.

FIG. 16 is a block diagram showing an image decoding apparatus whichrealizes an image decoding method according to the present embodiment.

An image decoding apparatus 200 according to the present embodimentincludes a variable length decoding unit 101, a motion compensation unit102, a picture memory 103, an adding unit 104, a conversion unit 105, amemory management unit 206, a picture number generation unit 207, and areproduction determination unit 108.

Note that, same reference numbers as used in the first embodiment areattached to indicate functional blocks of the image decoding apparatus200 of the present embodiment which are the same as those included inthe image decoding apparatus 100 according to the first embodiment, anddetailed explanations about the same functional blocks are omitted.

The picture number generation unit 207 obtains a reproductiondetermination signal from the reproduction determination unit 108,generates a picture number based on a reproduction format (normalconsecutive reproduction or random access reproduction) indicated by thereproduction determination signal, and outputs the generated picturenumber.

The memory management unit 206 manages the picture stored in the picturememory 103, using the reproduction determination signal outputted fromthe reproduction determination unit 108 and the picture number outputtedfrom the picture number generation unit 207.

FIG. 17 is a flowchart showing an example of operations by the imagedecoding apparatus 200.

First, when reproduction process of a stream Str is started, the imagedecoding apparatus 200 decodes a current picture to be decoded which islocated at a start position (Step S200), and generates a picture numberfor the current picture based on the generation method example 2 (StepS202). The image decoding apparatus 200 then assigns the generatedpicture number to the current picture, and stores the current pictureinto the picture memory 103 (Step S204). Next, the image decodingapparatus 200 determines whether or not random access reproduction isdesignated (Step S206), and when determining random access reproductionis designated (Y in Step S206), changes a picture management state usinga method which is different from that in the first embodiment (StepS208). The image decoding apparatus 200 then repeatedly executesprocesses from Step S200 to pictures as the random access reproductiondestinations. Specifically, the image decoding apparatus 200sequentially decodes each picture from the random access reproductiondestination when the changed management state as an initial state,stores pictures used as references into the picture memory 103, andmanages them.

On the other hand, when determining that random access reproduction isnot designated in Step S106 (N in Step S206), the image decodingapparatus 100 determines whether or not there is a not-yet-decodedpicture along with the sequence reproduction (Step S210). Whendetermining that there is a not-yet-decoded picture (Y in Step S210),the image decoding apparatus 200 repeatedly executes processes from StepS200 to the not-yet-decoded picture. On the other hand, when determiningthat there is no not-yet-decoded picture (N in Step S210), reproductionprocessing of the stream Str is terminated.

FIG. 18 is a flowchart which shows operations by the picture numbergeneration unit 207.

As shown in FIG. 18, the picture number generation unit 207 firstlyobtains a reproduction determination signal from the reproductiondetermination unit 108 (Step S220), and determines whether or not areproduction method to be designated is normal consecutive reproductionor random access reproduction from a random access point RAP of an OpenGOP structure, based on the reproduction determination signal (StepS222).

Here, when determining that normal consecutive reproduction isdesignated in Step S222, the picture number generation unit 207generates a picture number based on the generation method example 2(Step S224).

On the other hand, when determining that random access reproduction isdesignated in Step S222, the picture number generation unit 207initializes an STRL which is the reference picture list (Step S226).Specifically, the picture number generation unit 207 turns all picturesstored in the STRL which is the reference picture list to unreferencedpictures. For example, in the list management under the H.264 standard,the state of “STRL reference (used for short term reference)” is updatedto the state of “unreferenced (used for reference)”.

Due to the initialization of the STRL as a reference picture list, noreference picture is included in the STRL. Therefore, the picture numberPN takes its largest value when trick-play (random access reproduction)of the pictures Pic in the STRL is started. Note that, as described withreference to FIG. 9, when the picture number PN is the value closer tothe maximum picture number MPN, there is a possibility of causingmalfunction that the picture number PN exceeds the maximum picturenumber MPN as the sequence reproduction from the random accessreproduction position is continued. Specifically, in order to manageeach picture so as to prevent malfunction, the picture number PN may beset to a positive value closer to 0. In other words, the picture numberPN is generated based on the offset value FNO and the frame number fn sothat the offset value FNO may be reset to a small natural number.

Accordingly, the picture number generation unit 207 forcibly sets theoffset value FNO to 0 (Step S228). Note that, the offset value FNO maybe set to a value closer to the state taken when reproduction is startedfrom the Closed GOP, or a small value which is equal to or larger than0.

The picture management state is changed by such initialization of thereference picture list and setting of the offset value FNO=0.

The picture number generation unit 207 generates, for each picture fromthe random access point RAP, a picture number PN by adding and doublingthe offset value FNO and the frame number (a count value) fn of acurrent picture, based on the generation method example 2, with thechanged picture management state as an initial state (Step S230).

Note that, a stored value of the frame number fn immediately before whenrandom access reproduction is started may be changed to a value such as0. This aims to prevent that an addition between the offset value FNOand the maximum frame number MFN is caused and the both values of theoffset value FNO and the picture number PN are increased.

Accordingly, in the present embodiment, when random access reproductionis performed, the picture management state is changed by initializingthe reference picture list and setting the offset value so as to obtainFNO=0. Therefore, it can be prevented that the picture number PN asshown in FIG. 9 exceeds the maximum picture number. Consequently,occurrence of management instability based on the picture numbers PN ofpictures on the reference picture list is prevented, and as similar tothe case of normal consecutive reproduction, a picture to be decoded canbe decoded with reference to appropriate pictures using the relativeindexes Ind.

Note that, whereas the method of generating a picture number PN at thetime of random access reproduction is explained in the first and secondembodiments, if information regarding a Presentation Time Stamp (PTS)indicating a screen display time is included in a stream, it is possibleto generate an L0 list and an L1 list by diverting a PTS withoutswitching a method of generating a picture number PN between normalconsecutive reproduction and random access reproduction.

(First Variation)

Here, it shall be explained about variation of the reference picturelist management method according to the present embodiment.

FIG. 19 is a diagram for explaining a method of managing a referencepicture list (STRL) according to the present variation.

For example, it is assumed the case where stream reproduction as shownin FIG. 19( a) is performed. Respective pictures included in the streamare processed from the left to right direction in the diagram. In otherwords, time on the stream goes toward a direction from the left toright. Furthermore, head pictures of respective random access points RAPare pictures I0, I1, I2 and I3.

Here, after continuously reproducing pictures P4, P5, P6 and I7,rearward skipping operation (random access reproduction) whichsequentially reproduces a picture I3, a picture I2, a picture I1, and apicture I0 is performed.

In the present embodiment, when such random access reproduction isperformed, the previous reference picture lists are initialized. Inother words, all pictures listed in the reference picture list areturned into unreferenced pictures. The reference picture list is thenmanaged based on PTS so that the oldest PTS picture from among theunreferenced pictures is deleted from the reference picture list, and anew picture is inserted into the region from which said picture isdeleted.

Specifically, as shown in FIG. 19( b), the state of the referencepicture list becomes: a state a00 when pictures from the picture P4 tothe picture I7 are sequentially decoded; a state a01 when the picture I3is decoded; a state a02 when the picture I2 is decoded; a state a03 whenthe picture I1 is decoded; and a state a04 when the picture I0 isdecoded. It should be noted that the diagonally-shaded pictures in FIG.19 are unreferenced pictures that are retained as pictures only fordisplay.

In the reference picture list in the state a00, relative indexes Ind areassigned sequentially to the pictures I7, P6, P5 and P4. They aredisplayed on a screen in reverse order of the pictures P4, P5, P6 andP7. Each picture is a state of either “waiting to be displayed” or“displayed”.

In the reference picture list in the state a01, pictures I7, P6, P5 andP4 are determined as “unreferenced”, and a picture I3 is newly writtenover the picture P4 as a reference picture. Furthermore, respectivepictures are arranged sequentially in order of the pictures I3, P5, P6and I7, and displayed on the screen.

In the reference picture list in the state a02, the picture I3 ischanged to “unreferenced” picture, and a picture I2 is newly writtenover the picture I3 as a reference picture. Furthermore, respectivepictures are arranged sequentially in order of the pictures I2, P5, P6and I7, and displayed on the screen.

In the reference picture list in the state a03, the picture I2 ischanged to “unreferenced” picture, and a picture I1 is newly writtenover the picture I2 as a reference picture. Furthermore, respectivepictures are arranged sequentially in order of the pictures I1, P5, P6and I7, and displayed on the screen.

In the reference picture list in the state a04, the picture I0 ischanged to “unreferenced” picture, and a picture I0 is newly writtenover the picture I1 as a reference picture. Furthermore, respectivepictures are arranged sequentially in order of the pictures I0, P5, P6and I7, and displayed on the screen.

Accordingly, depending on a timing of displaying a picture on thescreen, the pictures P5, P6 and I7 are displayed, after the picture I0is reproduced through inverse reproduction. Furthermore, in the casewhere decoding processing is performed earlier than screen-displayingprocessing, there is a possibility that the pictures I3, I2, I1 and I0stored in the reference picture list cannot be displayed on the screenbecause they may be overwritten by subsequent decoded pictures beforethe screen-displaying.

Accordingly, in the present variation, when random access reproductionalong with an access to a random access point RAP of an Open GOP isperformed, occurrence of the aforementioned malfunction regardingscreen-displaying can be prevented by managing the reference picturelist without partially using the RTS information.

Specifically, in the present variation, a RAID number is updated everytime when random access reproduction is started, and the updated RAIDnumber is assigned to the picture to be stored into the referencepicture list. The reference picture list is then managed based on thePTS and the RAID number. Accordingly, as shown in FIG. 19( c), thepicture stored in the reference picture list is retained for a long termand occurrence of the aforementioned malfunction can be prevented.

FIG. 20A is a flowchart showing an operation of the memory managementunit 206 to assign a RAID number to an assign-target picture to whichthe RAID number is to be assigned.

First, the memory management unit 206 determines whether or not acurrent picture, which is a picture to be stored into the referencepicture list and to which a RAID number is to be assigned, is a firstpicture to be reproduced in reproduction processing (Step S300). Forexample, it determines whether or not all pictures on the referencepicture list are “unreferenced” pictures and only the displayed picturesare retained.

Here, when determining that the current picture is the first picture tobe reproduced in reproduction processing (Y in Step S300), the memorymanagement unit 206 determines the RAID number for the assign-targetpicture as 0 (Step S302). On the other hand, when determining that thecurrent picture is not the first picture to be reproduced in thereproduction processing (N in Step S300), the memory management unit 206further determines whether or not the assign-target picture is a firstpicture to be reproduced in random access reproduction (Step S304).

Here, when determining that the assign-target picture is a first pictureto be reproduced in the random access reproduction (Y in Step S304), thememory management unit 206 determines a value obtained by adding 1 tothe RAID number assigned previously to another picture as a RAID numberfor the assign-target picture (Step S306). On the other hand, whendetermining that the assign-target picture is not the first picture tobe reproduced in the random access reproduction (N in Step S304), thememory management unit 206 determines the RAID number assignedpreviously to another picture as a RAID number for the assign-targetpicture (Step S308).

As described in the above, the memory management unit 206 then assignsthe determined RAID number to the assign-target picture, and stores thepicture into the reference picture list (step S310).

FIG. 20B is a flowchart showing an operation of the memory managementunit 206 to select a picture to be deleted from the pictures stored inthe reference picture list.

First, the memory management unit 206 selects a picture group having thesmallest RAID number from among the pictures stored in the referencepicture list (Step S320).

Next, the memory management unit 206 selects a picture having the oldestPTS within the selected picture group as a picture to be deleted (StepS322).

It should be noted that actual process requires determination processessuch as whether or not each picture is a “reference” state or“unreferenced” state, and “waiting to be displayed” state or “displayed”state.

FIG. 19( c) is a diagram showing a state of reference picture listrealized through management by the memory management unit 206 using theRAID numbers as described in the above.

As shown in FIG. 19( c), the state of the reference picture listbecomes: a state a10 when pictures from the picture P4 to the picture I7are sequentially decoded; a state a11 when the picture I3 is decoded; astate a12 when the picture 12 is decoded; a state a13 when the pictureI1 is decoded; and a state a14 when the picture I0 is decoded. It shouldbe noted that diagonally-shaded pictures in FIG. 19 are unreferencedpictures.

In the reference picture list in the state a10, as similar to the caseof the reference picture list in the state a00, relative indexes Ind areassigned sequentially to the pictures I7, P6, P5 and P4. They aredisplayed on a screen in reverse order of the pictures P4, P5, P6 andP7. Each picture is in a state of “waiting to be displayed” or“displayed”. Furthermore, the RAID numbers assigned to these allpictures are 0.

In the reference picture list in the state a11, pictures I7, P6, P5 andP4 are turned into “unreferenced” pictures and a picture P4 is deleted,and a new picture I3 is added as a reference picture. Here, the RAIDnumbers of respective pictures I7, P6 and P5 are 0, and the RAID numberof the picture I3 is 1. Therefore, management based on the PTS is notcarried out among the pictures I7, P6 and P5 and the picture I3, but iscarried out only among the pictures I7, P6 and P5. Accordingly,respective pictures are arranged sequentially in order of the picturesP5, P6, I7 and I3, and displayed on the screen.

In the reference picture list in the state a12, the picture I3 is turnedinto an “unreferenced” picture and the picture P5 is deleted, and a newpicture I2 is added as a reference picture. Furthermore, respectivepictures are arranged sequentially in order of the pictures P6, I7, I3and I2, and displayed on the screen.

In the reference picture list in the state a13, the picture I2 is turnedinto an “unreferenced” picture, the picture P6 is deleted, and a newpicture I1 is added as a reference picture. Furthermore, respectivepictures are arranged sequentially in order of the pictures I7, I3, I2and I1, and displayed on the screen.

In the reference picture list in the state a14, the picture I1 is turnedinto an “unreferenced” picture and the picture I7 is deleted, and a newpicture I0 is added as a reference picture. Furthermore, respectivepictures are arranged sequentially in order of the pictures I3, I2, I1and I0, and displayed on the screen.

Accordingly, in the present variation, the reference picture list ismanaged based on the PTS and the RAID numbers. Therefore, pictures canbe retained in a necessary order for screen display.

It should be noted that, in the present variation, the offset value FNOdoes not need to be updated when random access reproduction isdesignated. Specifically, in the present variation, the referencepicture list may be initialized without updating the offset value FNO,and the reference picture list may be managed based on the PTS and theRAID numbers as described in the above.

(Second Variation)

Here, it shall be explained about another variation of the referencepicture list management method according to the present embodiment. Thefeature of the present variation is to complementarily change the numberof pictures managed by respective STRL and LTRL of the reference picturelists in accordance with a random access reproduction destination.

From FIG. 21A to FIG. 21F are diagrams, each showing a state ofreference picture list according to the present variation.

The reference picture list RPL includes an STRL which performs FIFO-typemanagement as described in the above, and an LTRL which carries outdeletion of a picture which is explicitly specified from the storage.

For example, as shown in FIG. 21A, the memory management unit 206configures the reference picture list RPL so that the total of 6pictures can be managed, stores four pictures of P4, P5, P6 and I7 intothe STRL, and stores two pictures of P0 and P1 into the LTRL.

Here, when random access reproduction is performed, the memorymanagement unit 206 firstly turns all pictures to be managed on thereference picture list RPL into unreferenced pictures. Specifically,pictures stored respectively in the LTRL and the STRL are set tounreferenced pictures.

Next, the memory management unit 206 tries to specify the number ofpictures to be managed in the STRL and the LTRL from the random accessreproduction destination. As a result, if knowing that the number ofpictures to be managed in the STRL is 4 pictures and the number ofpictures to be managed in the LTRL is 2 pictures from the randomreproduction destination, the memory management unit 206 retains thenumber of pictures managed respectively in the current STRL and LTRL.The memory management unit 206 then sequentially deletes unreferencedpictures stored in the STRL, and stores new four pictures of new0, new1,new 2 and new3 into the STRL as shown in FIG. 21C.

Here, if knowing that the number of pictures to be managed in the STRLis five pictures and the number of pictures to be managed in the LTRL isone picture as the random access reproduction destinations, the memorymanagement unit 206 changes the number of pictures managed respectivelyin the current STRL and LTRL in accordance with the random accessreproduction destination as shown in FIG. 21E. The memory managementunit 206 then stores five pictures of new0, new1, new2, new3, and new4into the STRL.

If the memory management unit 206 does not change the number of picturesmanaged in the STRL and LTRL in accordance with the random accessreproduction destination, the picture new0 is deleted in order to storethe picture new4 as shown in FIG. 21D. In such case, the picture new0cannot be referred to, and a possibility of causing malfunction in thedecoding processing is caused. Accordingly, in the present variation,occurrence of malfunction is prevented by changing the number ofpictures to be managed in the STRL and LTRL in accordance with therandom access reproduction destinations as described in the above.

Furthermore, in the case where knowing the number of pictures to bemanaged respectively by the STRL and LTRL from the random accessreproduction destination is respectively 6 pictures and 0 picture, thememory management unit 206 changes the number of pictures to be managedin the STRL to 6 pictures and the number of pictures to be managed inthe LTRL to 0 picture as shown in FIG. 21F. Note that, in the case whereit is necessary to store pictures into the LTRL after random accessreproduction is started, the pictures are stored after the number ofpictures to be managed in the STRL is reduced and the number of picturesto be managed in the LTRL is increased as special processing to beperformed when necessary.

Furthermore, in the case where the number of pictures to be managed arenot known as a result of trying to specify the number of pictures to bemanaged in the STRL and LTRL as the random access reproductiondestinations, the memory management unit 206 changes the number ofpictures to be managed in the STRL to the maximum 6 pictures and thenumber of pictures to be managed in the LTRL to the minimum 0 picture.It should be noted that, after the above process, the number of picturesto be managed in the STRL and the LTRL are updated every time when it isinstructed to store pictures into the LTRL, and the instructed picturesare stored. Such instruction to store pictures into the LTRL isdescribed in a Memory Management Control Operation (MMCO) in a headerinformation region of a picture, for example.

Accordingly, in the present variation, when random access to the OpenGOP is occurred, as special processing of managing the reference picturelist RPL to be performed when random access reproduction is occurred,the number of pictures to be managed respectively in the STRL and theLTRL are changed in accordance with the random access reproductiondestinations after not only setting all pictures in the STRL as“unreferenced” pictures but also setting the all pictures in the LTRL as“unreferenced” pictures.

It should be noted that, in the present variation, the offset value FNOdoes not need to be updated when random access reproduction isdesignated. Specifically, in the present variation, the referencepicture list may be initialized without updating the offset value FNO,and the number of pictures to be managed in the STRL and LTRL asdescribed in the above may be changed in accordance with the randomaccess reproduction destinations.

Note that, also in the present embodiment and respective variations, assimilar to the case of the first variation of the first embodiment, anempty region on the L0 list may be filled with a picture that is storedmost recently.

Note that, also in the present embodiment and respective variations, assimilar to the case of the second variation of the first embodiment,dummy pictures having picture numbers that are smaller than the picturenumber of the picture as the random access reproduction destination maybe inserted into the STRL and LTRL as many as the number of referencepermissible pictures.

Third Embodiment

Furthermore, the processes described in the aforementioned respectiveembodiments can be easily realized in an independent computer system byrecording a program for realizing the image decoding method mentioned inthe respective embodiments onto a recording medium such as a flexibledisk.

FIG. 22A to FIG. 22C are diagrams for explaining the case where imagedecoding method mentioned in the aforementioned respective embodimentsis realized in a computer system using a program recorded on a recordingmedium such as a flexible disc.

FIG. 22B is a front view and a schematic cross-section of a flexibledisk F as well as a front view of a flexible disk itself FD. FIG. 22Ashows an example of a physical format of the flexible disc FD as arecording medium body. The flexible disk F is configured by containingthe flexible disc FD into a case. A plurality of tracks Tr are formedconcentrically on the surface of the flexible disk in the radiusdirection from the periphery, and each track is divided into 16 sectorsSe in the angular direction. Therefore, in the flexible disk F storingthe above-mentioned program, the above program is recorded in an areaallocated for it on the above flexible disk FD.

In addition, FIG. 22C shows the configuration for recording andreproducing the program on and from the flexible disk F. When theprogram for realizing the image decoding method is recorded onto theflexible disk F, the computer system Cs writes the program onto theflexible disk FD via a flexible disk drive FDD. Furthermore, when theabove image decoding method is constructed in the computer system usingthe program recorded on the flexible disk F, the program is read outfrom the flexible disk F via the flexible disk drive FDD and transferredto the computer system.

Note that the above explanation is made on an assumption that arecording medium is a flexible disk, but the same processing can also beperformed using an optical disk. In addition, the recording medium isnot limited to these, but any other medium such as an IC card, and a ROMcassette can be used in the same manner if a program can be recordedonto them.

Fourth Embodiment

An AV processing apparatus according to the present embodiment performsapplied operations using the image decoding method described in thefirst or second embodiment.

FIG. 23 is a diagram showing a structure of an AV processing apparatusaccording to the present embodiment.

This AV processing apparatus 500 is configured as a DVD player, a BD-ROMplayer or the like which reproduces digital compressed audio and video.

A stream input/output unit 501 obtains a stream exStr from the outsideof the AV processing apparatus 500. A video coding/decoding unit 503performs coding and decoding on video. An audio coding/decoding unit 504performs coding and decoding on audio. A memory 505 holds data such asthe stream exStr, coded data, and decoded data. A memory input/outputunit 506 performs storage and extraction of data into and from thememory 505. A video processing unit 507 performs pre-processing andpost-processing on a video signal. A video input/output unit 508outputs, to the outside, a video signal as a video signal exVsig whichhas been processed by the video processing unit 507 or has passedwithout being processed by the video processing unit 507. Furthermore,the video input/output unit 508 captures the video signal exVSig fromthe outside.

The audio processing unit 509 also performs pre-processing andpost-processing on an audio signal. The audio input/output unit 510outputs, to the outside, an audio signal as an audio signal exASig whichhas processed by the audio processing unit 509 or has passed withoutbeing processed by the audio processing unit 509. The audio input/outputunit 510 also captures the audio signal exASig from the outside. The AVcontrol unit 511 controls the entire AV processing apparatus 500.

The stream input/output unit 501, the picture coding/decoding unit 503,the audio coding/decoding unit 504, a memory input/output unit 506, avideo processing unit 507, an audio processing unit 509, and an AVcontrol unit 511 are connected to the bus 502 which transfers the streamexStr and data such as decoded audio/video data.

Here, it shall be described about the decoding operations performed bysuch AV processing apparatus 500. First, a stream input/output unitexStrIF obtains an audio/video stream exStr through reading out of datafrom a storage medium such as an optical disk, and data reception fromsatellite broadcast and the like. The video coding/decoding unit 503extracts a video stream from the stream exStr, and the audiocoding/decoding unit 504 extracts an audio stream from the stream exStr.

The memory input/output unit 506 stores, into the memory 505, video datadecoded by the video coding/decoding unit 503. The video processing unit507 extracts data stored in the memory 505, and performs processing suchas noise removal and the like. Furthermore, the video coding/decodingunit 503 extracts the video data stored in the memory 505 again, anduses as a reference picture for inter-picture motion compensationprediction.

Here, the video coding/decoding unit 503 includes the image decodingapparatus 100 or 200 of the first embodiment or the second embodiment.It should be noted that the video coding/decoding unit 503 does not needto include the picture memory 103 of the image decoding apparatuses 100and 200. In this case, the video coding/decoding unit 503 uses a memory505 in place of the picture memory 103. Furthermore, the above videostream corresponds to the stream Str of the first embodiment or thesecond embodiment.

On the other hand, the memory input/output unit 506 stores, into thememory 505, audio data decoded by the audio coding/decoding unit 504.The audio processing unit 509 obtains data stored in the memory 505, andperforms processing relating to sound and the like on the obtained data.

Lastly, in temporal synchronization between audio and video, the dataprocessed by the video processing unit 507 is outputted as a videosignal exVSig via the video input/output unit 508 and displayed on atelevision screen and the like. Furthermore, the data processed by theaudio processing unit 509 is outputted as an audio signal exASig via theaudio input/output unit 510, and outputted from a speaker and the like.

It should be noted that respective functional blocks included in theimage decoding apparatus 100 or 200 of the first embodiment or thesecond embodiment is basically realized as a Large Scale Integration(LSI) that is an integrated circuit. These elements may be separatelyintegrated into one chip, or may be integrated so that only a portion ofthese elements or all of these elements are included. (For example,functional blocks other than memory such as the picture memory 103 maybe integrated into one chip.)

Furthermore, whereas an LSI is used as the integrated circuit, it may berealized as an Integrated Circuit (IC), a system LSI, a super LSI, andan ultra LSI. In addition, a method of constructing an integratedcircuit is not limited to the LSI and may be realized as a privatecircuit or as a general processor. After the LSI is manufactured, aField Programmable Gate Array (FPGA) capable of programming, and areconfigurable processor which can reconfigure connection and setting ofcircuit cells in the LSI may be used. Furthermore, there is no doubtthat, if technology of constructing an integrated circuit is introducedas a replace of the LSI by a development of semiconductor technology orderived another technology, the functional blocks may be integratedusing such technology. There is a possibility to apply biotechnology andthe like.

Furthermore, only a memory (a picture memory 103 and a memory 505) whichholds data to be decoded, from among the functional blocks, may beseparately constructed without being integrated into one chip.

INDUSTRIAL APPLICABILITY

The image decoding method of the present invention has an effect ofcapable of decoding a picture to be decoded with reference toappropriate pictures even when random access reproduction is performed,and is applicable to, for example, a DVD player, a BD-ROM player and thelike which reproduce video and audio using H.264 standard.

1. An image decoding method for decoding each of coded pictures in astream with reference to a picture that is previously decoded and storedin a memory, said method comprising: a first storage step ofsequentially decoding each of the coded pictures in the stream, storing,into the memory, a picture to be used as a reference from among thedecoded pictures, and managing the picture stored in the memory; adetermination step of determining whether or not random accessreproduction has been designated; a state change step of changing amanagement state for managing the picture in the memory in the casewhere it is determined that random access reproduction has beendesignated in said determination step; and a second storage step ofsequentially decoding each of the coded pictures from a random accessreproduction destination with the management state changed in said statechange step as an initial state, storing, into the memory, a picture tobe used as a reference from among the decoded pictures, and managing thepicture stored in the memory.
 2. The image decoding method according toclaim 1, wherein the coded pictures in the stream are divided intogroups on an access unit basis, and in said determination step, it isdetermined whether or not random access reproduction starting from anaccess unit which is allowed to refer to a picture of another accessunit has been designated.
 3. The image decoding method according toclaim 2, wherein in respective said first and second storage steps, thepicture to be used as a reference is managed by generating picturenumbers for identifying each of the decoded pictures and assigning apicture number to each of the decoded pictures, and in said state changestep, an offset value for generating the picture number is updated asthe management state.
 4. The image decoding method according to claim 3,wherein in said first storage step, the offset value is updated so thatthe offset value is increased every time when all the pictures includedin a current access unit to be decoded are decoded, and a picture numberfor each of the decoded pictures is generated using the offset value anda count value assigned to each of the decoded pictures in the stream onan access unit basis, in said state change step, the offset value isupdated so that the offset value is increased, and in said secondstorage step, with the offset value updated in said state change step asan initial state, the offset value is updated so that the offset valueis increased every time when all the pictures in a current access unitto be decoded are decoded, and a picture number for each of the decodedpictures from the random access reproduction destination is generatedusing the count value and the offset value.
 5. The image decoding methodaccording to claim 4, wherein in respective said first and secondstorage steps and said state change step, the offset value is updated sothat the offset value is increased at least by the maximum value of acount value assigned on an access unit basis.
 6. The image decodingmethod according to claim 2, wherein in respective said first and secondstorage steps, the picture to be used as a reference is managed bygenerating picture numbers for identifying each of the decoded picturesand assigning a picture number to each of the decoded pictures, and insaid state change step, the management state is updated by changing allthe pictures stored in the memory to unreferenced pictures.
 7. The imagedecoding method according to claim 6, wherein in said state change step,all the pictures stored in the memory are changed to unreferencedpictures while all the pictures are kept displayable.
 8. The imagedecoding method according to claim 7, wherein in respective said firstand second storage steps, the picture to be used as a reference ismanaged by generating picture numbers for identifying each of thedecoded pictures and assigning a picture number to each of the decodedpictures, in said first storage step, an offset value for generating apicture number is updated so that the offset value is increased everytime when a count value of a current picture to be decoded is smallerthan a count value of a picture decoded immediately before the currentpicture, and a picture number for each of the decoded pictures isgenerated using the offset value and the count value assigned to each ofthe decoded pictures in the stream on an access unit basis, in saidstate change step, the offset value existing as the management state isupdated so that the offset value is decreased, in said second storagestep, with the offset value updated in said state change step as aninitial state, the offset value is updated so that the offset value isincreased, every time when a count value of a current picture to bedecoded is smaller than a count value of a picture decoded immediatelybefore the current picture, and a picture number for each of the decodedpictures from the random access reproduction destination is generatedusing the count value and the offset value.
 9. The image decoding methodaccording to claim 8, wherein in said state change step, the offsetvalue is updated to
 0. 10. The image decoding method according to claim7, wherein further in said second storage step, a picture to be storedinto the memory is managed using start information based on a start ofrandom access reproduction.
 11. The image decoding method according toclaim 10, wherein in said second storage step, a random access numberwhich is to be updated every time when random access reproduction isstarted is used as the start information.
 12. The image decoding methodaccording to claim 7, wherein the memory includes a first storage regionand a second storage region, each having a different picture managementmode, and further in said state change step, a size of the first storageregion and a size of the second storage region are changedcomplementarily according to a random access destination.
 13. The imagedecoding method according to claim 12, wherein, in the first storageregion, an oldest picture which has been stored for a longest time isdeleted when a new picture is stored, and in the second storage region,a designated picture is deleted when a new picture is stored.
 14. Theimage decoding method according to claim 13, wherein in said statechange step, it is determined whether or not the size of the firststorage region and the size of the second storage region to be setaccording to a random access reproduction destination can be specified,and when it is determined that the sizes cannot be specified, allregions in the memory are allocated to the first storage region.
 15. Theimage decoding method according to claim 6, wherein further in saidsecond storage step, in the case where there is an unreferenced picturein the memory, the unreferenced picture is replaced with a referencepermissible picture which is stored most recently.
 16. The imagedecoding method according to claim 2, wherein in respective said firstand second storage steps, the picture to be used as a reference ismanaged by generating picture numbers for identifying each of thedecoded pictures and assigning a picture number to each of the decodedpictures, and in said state change step, a picture number of a picturestored in the memory is changed to a picture number which is smallerthan a picture number of a picture from the random access reproductiondestination.
 17. The image decoding method according to claim 16,wherein in said state change step, the picture stored in the memory isreplaced with a dummy picture having a picture number which is smallerthan the picture number of the picture from the random accessreproduction destination.
 18. The image decoding method according toclaim 17, wherein the dummy picture is a picture which is not includedin the stream.
 19. The image decoding method according to claim 17,wherein the dummy picture is a picture decoded in said first storagestep.
 20. An image decoding apparatus which decodes each of codedpictures in a stream with reference to a picture that is previouslydecoded and stored in a memory, said image decoding apparatuscomprising: a first storage unit operable to sequentially decode each ofthe coded pictures in the stream, store, into the memory, a picture tobe used as a reference from among the decoded pictures, and to managethe picture stored in the memory; a determination unit operable todetermine whether or not random access reproduction has been designated;a state change unit operable to change a management state for managingthe picture in the memory, in the case where it is determined thatrandom access reproduction has been designated by said determinationunit; and a second storage unit operable to sequentially decode each ofthe coded pictures from a random access reproduction destination withthe management state changed by said state change unit as an initialstate, store, into the memory, a picture to be used as a reference fromamong the decoded pictures, and to manage the picture stored in thememory.
 21. An integration circuit which decodes each of coded picturesin a stream with reference to a picture that is previously decoded andstored in a memory, said integration circuit comprising: a first storageunit operable to sequentially decode each of the coded pictures in thestream, store, into the memory, a picture to be used as a reference fromamong the decoded pictures, and to manage the picture stored in thememory; a determination unit operable to determine whether or not randomaccess reproduction has been designated; a state change unit operable tochange a management state for managing the picture in the memory, in thecase where it is determined that random access reproduction has beendesignated by said determination unit; and a second storage unitoperable to sequentially decode each of the coded pictures from a randomaccess reproduction destination with the management state changed bysaid state change unit as an initial state, store, into the memory, apicture to be used as a reference from among the decoded pictures, andto manage the picture stored in the memory.
 22. A program for decodingeach of coded pictures in a stream with reference to a picture that ispreviously decoded and stored in a memory, said program causing acomputer to execute: a first storage step of sequentially decoding eachof the coded pictures in the stream, storing, into the memory, a pictureto be used as a reference from among the decoded pictures, and managingthe picture stored in the memory; a determination step of determiningwhether or not random access reproduction has been designated; a statechange step of changing a management state for managing the picture inthe memory in the case where it is determined that random accessreproduction has been designated in said determination step; and asecond storage step of sequentially decoding each of the coded picturesfrom a random access reproduction destination with the management statechanged in said state change step as an initial state, storing, into thememory, a picture to be used as a reference from among the decodedpictures, and managing the picture stored in the memory.